Application specific schema extensions for a hierarchical data structure

ABSTRACT

A schema for a hierarchical data structure may include application specific extensions to the schema applied to a hierarchical data structure. Class may be added to the schema by individual applications granted access to a hierarchical data structure. When an access request for an object of the hierarchical data structure is received, the class may be identified in the schema and applied to process the access request to the object. Different classes may be added by different applications without disrupting the utilization of the schema for accessing the hierarchical data structure of other applications.

This application is a continuation of U.S. patent application Ser. No.15/475,020, filed Mar. 30, 2017, which is hereby incorporated byreference herein in its entirety.

BACKGROUND

Hierarchical storage systems offer users a way to persist hierarchicaldata structures. Hierarchical data structures stored in suchhierarchical storage systems may provide users with the ability to storeand managed related data in way that reflects the relationships betweenthe data. Such hierarchical structures may be used to implement manydifferent services, including resource management systems, directorysystems, payroll systems, and other systems that leverage datarelationships to reason over the data stored therein. Techniques thatoptimize interactions with hierarchical storage systems, reducing userburden and providing greater management flexibility may be highlydesirable, such as techniques that increase the consistency with whichmultiple applications can access the same hierarchical data structure.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A-1C are logical block diagrams illustrating application specificextensions to a schema for a hierarchical data structure, according tosome embodiments.

FIG. 2 is a logical block diagram illustrating a provider network thatimplements a directory storage service including a hierarchical datastore that implements application specific extensions to a schema for ahierarchical data structure, according to some embodiments.

FIG. 3 is a logical block diagram illustrating a hierarchy storage node,according to some embodiments.

FIG. 4 is a logical block diagram illustrating a data model for ahierarchal data structure in a hierarchical data store, according tosome embodiments.

FIG. 5 is a logical block diagram illustrating a data model for a schemaapplicable to a hierarchal data structure in a hierarchical data store,according to some embodiments.

FIG. 6 is a logical block diagram illustrating interactions to manage aschema applicable to a hierarchical data structure, according to someembodiments.

FIG. 7 is a logical block diagram illustrating interactions to access anobject in a hierarchical data structure that has application specificextensions to the schema, according to some embodiments.

FIG. 8 is a high-level flowchart illustrating methods and techniques toprovide access to a hierarchical data structure with applicationspecific extensions to a schema for the hierarchical data structure,according to some embodiments.

FIG. 9 is a high-level flowchart illustrating methods and techniques toadd a class to a schema that extends the schema for an application thataccesses a hierarchical data structure, according to some embodiments.

FIG. 10 is a high-level flowchart illustrating methods and techniques toupdate an object based on a tag, according to some embodiments.

FIG. 11 is a high-level flowchart illustrating methods and techniques tocreate an object in a hierarchical data structure according to a tag,according to some embodiments.

FIG. 12 is an example computer system, according to various embodiments.

While embodiments are described herein by way of example for severalembodiments and illustrative drawings, those skilled in the art willrecognize that the embodiments are not limited to the embodiments ordrawings described. It should be understood, that the drawings anddetailed description thereto are not intended to limit embodiments tothe particular form disclosed, but on the contrary, the intention is tocover all modifications, equivalents and alternatives falling within thespirit and scope as defined by the appended claims. The headings usedherein are for organizational purposes only and are not meant to be usedto limit the scope of the description or the claims. As used throughoutthis application, the word “may” is used in a permissive sense (i.e.,meaning having the potential to), rather than the mandatory sense (i.e.,meaning must). The words “include,” “including,” and “includes” indicateopen-ended relationships and therefore mean including, but not limitedto. Similarly, the words “have,” “having,” and “has” also indicateopen-ended relationships, and thus mean having, but not limited to. Theterms “first,” “second,” “third,” and so forth as used herein are usedas labels for nouns that they precede, and do not imply any type ofordering (e.g., spatial, temporal, logical, etc.) unless such anordering is otherwise explicitly indicated.

Various components may be described as “configured to” perform a task ortasks. In such contexts, “configured to” is a broad recitation generallymeaning “having structure that” performs the task or tasks duringoperation. As such, the component can be configured to perform the taskeven when the component is not currently performing that task (e.g., acomputer system may be configured to perform operations even when theoperations are not currently being performed). In some contexts,“configured to” may be a broad recitation of structure generally meaning“having circuitry that” performs the task or tasks during operation. Assuch, the component can be configured to perform the task even when thecomponent is not currently on. In general, the circuitry that forms thestructure corresponding to “configured to” may include hardwarecircuits.

Various components may be described as performing a task or tasks, forconvenience in the description. Such descriptions should be interpretedas including the phrase “configured to.” Reciting a component that isconfigured to perform one or more tasks is expressly intended not toinvoke 35 U.S.C. § 112(f), interpretation for that component.

“Based On.” As used herein, this term is used to describe one or morefactors that affect a determination. This term does not forecloseadditional factors that may affect a determination. That is, adetermination may be solely based on those factors or based, at least inpart, on those factors. Consider the phrase “determine A based on B.”While B may be a factor that affects the determination of A, such aphrase does not foreclose the determination of A from also being basedon C. In other instances, A may be determined based solely on B.

The scope of the present disclosure includes any feature or combinationof features disclosed herein (either explicitly or implicitly), or anygeneralization thereof, whether or not it mitigates any or all of theproblems addressed herein. Accordingly, new claims may be formulatedduring prosecution of this application (or an application claimingpriority thereto) to any such combination of features. In particular,with reference to the appended claims, features from dependent claimsmay be combined with those of the independent claims and features fromrespective independent claims may be combined in any appropriate mannerand not merely in the specific combinations enumerated in the appendedclaims.

DETAILED DESCRIPTION

Various embodiments of application specific extensions to a schema for ahierarchical data structure are described herein. A hierarchical datastore, such as a directory data store, may allow users toprogrammatically manage data using client-side applications that accessdata stored in the hierarchical data store in order to perform differentoperations. Some data stored in the hierarchical data structure may beuseful to multiple different applications. However, applications mayalso rely upon data that is specific to the operations of an individualapplication. A schema for the hierarchical data structure may define orotherwise describe the data that is maintained for a hierarchical datastructure. In order to allow applications to add further data fordifferent operations, changes to the schema may be coordinated.Application specific extensions to a schema may allow schema changes tobe made without interfering with the performance of other applications.For example, one or more classes may be added to a schema on behalf ofone application in order to provide specific data to be used by theapplication without modifying other classes or data in the schema thatare utilized by other applications, allowing for the independentdevelopment and operation of applications that access the samehierarchical data structure. Moreover, the additions to the schema maybecome useful to other applications over time, and thus may beaccessible to the other applications, in some embodiments.

FIG. 1A is a logical block diagram illustrating application specificextensions to a schema for a hierarchical data structure, according tosome embodiments. A hierarchical data structure, such as hierarchicaldata structure 120 may be stored as part of a hierarchical data store110. A hierarchical data structure may include one or multiple objectsorganized according to one or multiple links that provide relationships,paths or other form hierarchy between objects. In this way, therelationship of objects, and data values associated or stored as part ofthe objects, can be modeled and maintained in the data structure. Forexample, an organization chart indicating the reporting structurebetween company managers and employees can be modeled in a hierarchicaldata structure that indicates the relationship between employees, andmay include data specific to each employee (e.g., name, assignment,years of service, etc.).

In at least some embodiments, a schema, such as scheme 130, may beapplied to a hierarchical data structure to further define the differentrelationships between objects. For example, each employee may be aninstance of an employee class defined in a schema for the hierarchicaldata structure, some employees may also be instances of a manager class,and some employees may instances of an executive class. Each class mayinclude attributes and other values which may be automatically allocatedto (or defined for) an object that is the instance of that class. Forexample, the executive class may include an attribute for managedbusiness units. When creating or updating an object of the executiveclass managed business unit values can be stored. A client applicationaccessing the hierarchical data structure can understand from the schemathat an executive object is expected to (or possibly can) have valuesfor managed business units.

Multiple applications 140 may be granted access to the same hierarchicaldata structure 120. For example, each application may request, register,associate, apply, or otherwise gain permission and access credentials toaccess hierarchical data structure 120. Each of the applications 140 mayhave access to data stored in hierarchical data structure according toschema 130. Some portions of schema 130 may be common portions (e.g.,portions that exist when applications first gain access to thehierarchical data structure), while some portions may be created fordifferent applications, such as application class extensions 132 and134, in order to enhance, add, or otherwise adapt the data that ismaintained in the hierarchical data structure 120 for different usecases, operations, or scenarios that applications 140 may perform orexperience. For example, if hierarchical data structure stores employeedirectory data, such as name, title, location, then differentapplications may be developed to interact with the employee data butinclude further information for different operations. A vacationmanagement tool, for example, may store available paid time off or sickdays for individual employees, as well as accrual rates for additionalpaid time off or sick days. Another application, such as a humanresource application, can track salary information, management orreporting information, performance reviews, salary data, etc.

Application class extensions can be applied to allow each application tospecify additional data (or other class provided functionality) tomaintain in order to facilitate the operations of that application, invarious embodiments. This application specific data may then remainisolated from other applications (unless permission is granted), in oneembodiment. While in another embodiment, the application specific datacreated by the class may be automatically shared or made available toanother application. Creation of application specific class extensionsmay be performed without any coordination amongst applications 140, sothat development of applications 140 can proceed independently (e.g.,including the ability to add additional features and correct bugswithout affecting the operations of other applications accessinghierarchical data structure 120.) For example, schema class extensionsmay be validated prior to inclusion in the schema, as discussed belowwith regard to FIG. 9, so that inclusion of a class does not break,block, or otherwise disrupt the operation of other applications that canaccess the hierarchical data structure. In another example, data values,information or other attributes that are specific to a particularapplication may only be visible to that application (e.g., according topermissions or other information included in a definition for anapplication-specific extension). In this way, schema extensionattributes can be isolated from changes made by other applications andprivate.

As illustrated in FIG. 1A, object 122 includes common schema attributes152 and schema extension attributes 154 and 156. Common schemaattributes 152 may be data that is accessible and visible to anyapplication 140. However, schema extension attributes may be limited toa particular application for which the schema was extended. Forinstance, in FIG. 1B, application 140 a may access 162 object 122. Inaddition to common schema attributes 152, schema extension attributes154 may be visible to application 140 a, as the schema extensiondefining schema extension attributes 154 may be for application 140 a.Likewise, in FIG. 1C, application 140 b may access 164 object 122. Inaddition to common schema attributes 152, schema extension attributes156 may be visible to application 140 b, as the schema extensiondefining schema extension attributes 154 may be for application 140 b.In this way, both applications 140 a and 140 b can safely interact withthe same hierarchical data structure, customizing the data maintained inthe hierarchical data structure, without interfering with the schema orperformance of the other application. Note that in other embodiments,schema class extensions may be publicly available for read access toother applications. Thus, application 140 b, for instance, may be ableto write or otherwise update schema extension attributes 154 and read(but not update) schema extension attributes 156.

Please note, FIGS. 1A-1C are provided as a logical illustration of ahierarchical data structure with application specific extensions for aschema for the hierarchical data structure, and is not intended to belimiting as to the physical arrangement, size, or number of components,modules, or devices, implementing applications, a hierarchical datastore, schema, or hierarchical data structure.

The specification first describes an example of a distributedhierarchical data store as a network-based directory storage service,according to various embodiments. The example directory storage servicemay store hierarchical data structures for many different clients, invarious embodiments. Included in the description of the exampledirectory storage service are various aspects of the example directorystorage service along with the various interactions between thedirectory storage service and clients. The specification then describesa flowchart of various embodiments of methods for application specificextensions to a schema for a hierarchical data structure. Next, thespecification describes an example system that may implement thedisclosed techniques. Various examples are provided throughout thespecification.

FIG. 2 is a logical block diagram illustrating a provider network thatimplements a directory storage service including a hierarchical datastore that implements application specific extensions to a schema for ahierarchical data structure, according to some embodiments. Providernetwork 200 may be set up by an entity such as a company or a publicsector organization to provide one or more services (such as varioustypes of cloud-based computing or storage) accessible via the Internetand/or other networks to clients 210. Provider network 200 may includenumerous data centers hosting various resource pools, such ascollections of physical and/or virtualized computer servers, storagedevices, networking equipment and the like (e.g., computing system 2000described below with regard to FIG. 12), needed to implement anddistribute the infrastructure and services offered by the providernetwork 200. In some embodiments, provider network 200 may implement adirectory storage service 220, described, provide other computingresources or services, such as a virtual compute service and storageservices, such as object storage services, block-based storage services,data warehouse storage service, archive storage service 270 and/or anyother type of network based services 280 (which may include variousother types of storage, processing, analysis, communication, eventhandling, visualization, and security services, such as a code executionservice that executes code without client management of the executionresources and environment). Clients 210 may access these variousservices offered by provider network 200 via network 260. Likewisenetwork-based services may themselves communicate and/or make use of oneanother to provide different services. For example, various ones ofother service(s) 280 may store, access, and/or rely upon hierarchicaldata structures stored in directory storage service 220 or may executecode associated with an object in a hierarchical data structure storedin directory storage service 220.

In various embodiments, the components illustrated in FIG. 2 may beimplemented directly within computer hardware, as instructions directlyor indirectly executable by computer hardware (e.g., a microprocessor orcomputer system), or using a combination of these techniques. Forexample, the components of FIG. 2 may be implemented by a system thatincludes a number of computing nodes (or simply, nodes), each of whichmay be similar to the computer system embodiment illustrated in FIG. 12and described below. In various embodiments, the functionality of agiven service system component (e.g., a component of the databaseservice or a component of the storage service) may be implemented by aparticular node or may be distributed across several nodes. In someembodiments, a given node may implement the functionality of more thanone service system component (e.g., more than one database servicesystem component).

Directory storage service 220 may store, manage, and maintainhierarchical data structures, such as a directory structure discussedbelow with regard to FIG. 4, stored at various ones of storage node(s)240 (in single tenant or multi-tenant fashion). Clients of directorystorage service 220 may operate on any subset or portion of thehierarchical data structure with transactional semantics and/or mayperform path-based traversals of hierarchical data structures. Suchfeatures allow clients to access hierarchical data structures in manyways. For instance, clients may utilize transactional access requests toperform multiple operations concurrently, affecting different portions(e.g., nodes) of the hierarchical directory structure (e.g., readingparts of the hierarchical directory structure, adding an object, andindexing some of the object's attributes, while imposing the requirementthat the resulting updates of the operations within the transaction areisolated, consistent, atomic and durably stored).

In various embodiments, directory storage service 220 may implementrouting layer 232 to direct access requests from internal or externalclients to the appropriate storage node(s) 240. For example, routinglayer 232 may implement a fleet of routing nodes that maintain mappinginformation which identifies the locations of a hierarchical datastructures on storage node(s) 240. When an access request is received,routing layer nodes may then determine which one of the storage node(s)that hosts the hierarchical data structure identified in the accessrequest to send the access request. Consider a scenario wherehierarchical data structures may be replicated across multiple differentstorage nodes 240 as part of a replica group. Routing 232 may implementvarious load balancing schemes to direct requests from different clientsto different hierarchy storage nodes within the replica group, so thatno single storage node becomes overburdened. Moreover, as storage nodes240 may utilize tokens to maintain state across different accessrequests sent by clients so that different hierarchy storage node(s) 240may handle each request from the client, routing 232 need not trackwhich storage node is communicating with which client.

Control plane 234 may implement various control functions to manage thestorage node(s) 240 and other components of directory storage service220 that provide storage of hierarchical data structures, such asdirecting creation and placement of new hierarchical data structures onstorage node(s) 240, storage scaling, heat management, node repairand/or replacement. For example, various placement schemes may utilizetechniques such as consistent hashing (e.g., based on hashing anidentifier for individual hierarchical data structures) to identifystorage node(s) to store versions of the hierarchical data structure, orrandomly mapping hierarchical data structures to a number hierarchystorage node(s) 240 that form a replica set. To provide heat management,for example, control plane 234 may collect storage node(s) 240 metricspublished by each node. Each host may have various thresholds forperformance characteristics, such as memory utilization, CPUutilization, disk utilization, and request-rate capacity. When ahierarchy storage node reports metrics that exceed a threshold (ormultiple thresholds), control plane 234 may direct the migration of oneor more hierarchical data structures to different storage nodes.Similarly, control plane 234 may detect when certain storage nodes areunable to keep up with access requests directed to a particular replicagroup for a hierarchical data structure and may provision additionalstorage node(s) to horizontally scale the replica group to better meetthe access request demand.

Storage node(s) 240 may maintain and handle access to hierarchicalstorage nodes in directory storage service 220. FIG. 3 is a blockdiagram illustrating a hierarchy storage node, according to someembodiments. Hierarchy storage node 300 may implement request handler310 to process access requests and pass along appropriate instructionsor requests to other components, such as storage engine 340, transactionlog interface 350 or archive interface 360. For example, access requesthandler 310 may interpret various requests formatted according to aprogrammatic interface, such as an application programming interface(API). Access requests may include various, such as various accessrequests to create, update, attach, detach, delete and query objects ina hierarchical data structure, and access requests to define, populate,discover, and query a local index (which may be strongly consistent andmaintained as part of or separately from the hierarchical datastructure) on hierarchical data structure object attributes, asdiscussed below with regard to FIG. 7.

In various embodiments, storage engine 340 may be a storage engine thatinteracts with structure or format of data as it is stored inhierarchical data structure store 320 (e.g., a key-value storage enginefor data maintained in key-value storage format, relational data storageengine for data maintained in a relational storage format, etc.) andschema store 330 which may maintain a schema that is applied to thehierarchical data structure maintained in hierarchical data structurestore 320, which may be maintained according to the models discussedbelow with regard to FIGS. 4 and 5. In some embodiments, hierarchicaldata structure store 320 may be partially or completely implemented inmemory or other quick access storage devices, such as random accessmemory devices (RAM), as well as utilizing persistent block-basedstorage devices to store historical hierarchical data structure 330,including magnetic disk or solid state drives. In some embodiments,caching techniques may be implemented so that frequently accessedportions of data, such as frequently access portions of hierarchicaldata structures are maintained in memory components whereas otherportions are maintained in block-based persistent storage components.Storage node 300 may operate multi-tenant storage for hierarchical datastructures so that different hierarchical data structures maintained onbehalf of different clients, accounts, customers, and the like may bemaintained in hierarchical data structure store 320 and schemas appliedto the hierarchical data structures in schema store 330. For example,storage node 300 may participate in different replica groups withdifferent storage nodes for the different hierarchical data structuresstored at storage node 300.

Transaction log interface 350 may provide capabilities to interact with(e.g., validate transactions) with respect to the logs corresponding tohierarchical data structures stored in transaction log storage 250 forthe hierarchical data structures, according to the various techniques.Transaction log storage 250 may provide consistent storage for versionedhierarchical data structures, according to some embodiments. Multipleclients may perform various access requests to a hierarchical datastructure concurrently, such as various write requests. In a least someembodiments, a replica group may include multiple storage nodes thatmaintain versions of the hierarchical data structure that are availablefor servicing various access requests from the multiple clients. Forexample, clients can submit different write requests to storage nodesaccording to a routing layer (e.g., routing layer 232) which may directaccess requests from each client to a different storage node in thereplica group according to a load balancing scheme. Upon receiving therequest, each storage node may perform various operations upon a versionof the hierarchical data structure at the storage node, then offer thewrites to transaction log storage 210 for commitment to directorystructure log stored in transaction log storage (including variousinformation such as the affected or accessed data by performing thewrite request, the write request itself, and a transaction sequencenumber of other indication identifying the point-in-time of the currentversion of the hierarchical data structure at the storage node).Indications of commitment or conflict may be provided back fromtransaction log storage 250 to the respective storage nodes. For thosewrites that are committed, the directory structure log may be read andcommitted writes applied to the respective versions of the hierarchicaldata structure maintained at the multiple storage nodes.

Similarly, archive interface 360 may be implemented to retrieve archivedtransactions or snapshots to service an access request for historicalchanges to the hierarchical data structure, a historical query, or otheraccess requests that require a version of the hierarchical datastructure that is older than that maintained in hierarchical datastructure store 320. In some embodiments, an archival management layer238 may utilize transactions stored for different hierarchical datastructures stored in respective transaction logs in transaction logstorage 250 to generate and store snapshots of the hierarchical datastructure at different points in time in archive storage service 270.For example, archival management 238 may determine when snapshots of ahierarchical data structure should be captured, provision appropriatestorage locations in archive storage service 270, and direct archiveworker nodes (not illustrated) to perform the read, write, and otheroperations to generate and place the snapshots in archive storageservice 270. Similarly, archival management 238 may direct the copyingand storage of individual log records/transactions and/or groups of logrecords and transactions to be stored as part of an archived transactionlog for hierarchical data structures in archive storage service 270.Archival management 238 may read the hierarchical data structure log toretrieve writes for transmission as archived transactions or snapshotsof the hierarchical data structure maintained in archive storage service270. Archival management 238 may then periodically or aperiodicallyupdate an archived log in archive storage service 270 and generate andsend new snapshots to be maintained as part of archived snapshots inarchive storage service 270. In this way, the hierarchical datastructure can be recreated at any point-in-time, for example by loadinga snapshot onto a storage node and applying transactions from anarchived log to reach a certain transaction sequence number so that theversion of the hierarchical data structure at the storage number isconsistent with a specified point-in-time.

Turning back to FIG. 2, transaction log storage 250 may provide a faulttolerant, high performance, durable, log publishing service. Transactionlog storage 250 may be used as a commit log underlying stronglyconsistent distributed applications such as databases, key-value stores,and lock managers, and as illustrated in FIG. 2 directory storageservice 220 providing hierarchical data storage, as discussed above.Transaction log storage 250 may provide strong consistency guaranteesand support constraints between committed records, to enable featureslike deduplication, sequencing, and read-write conflict detection. Forexample, in various requests transaction log storage 250 may determinewhether or not to commit changes to hierarchical data structures (e.g.,write requests and other modifications) by examining a proposedtransaction for conflicts with other committed transactions. Such afeature may provide a fine-grained locking model over the hierarchicaldata structure (e.g., only those portions of the hierarchical datastructure affected by a conflict between transactions may be locked).Transaction log storage 250 may maintain a separate log or chain of logrecords for each hierarchical data structure, serving as anauthoritative definition of the changes to the state hierarchical datastructure over time. Transactions may be ordered according totransaction sequence numbers, which may be monotonically increasing toreference the state of a hierarchical data structure at individualpoints in time. Note that in some embodiments, transaction log storage250 may be a separate network-based storage service implemented as partof provider network 250 external to directory storage service 220.

Generally speaking, clients 210 may encompass any type of clientconfigurable to submit network-based services requests to providernetwork 200 via network 260, including requests for directory services(e.g., a request to create or modify a hierarchical data structure to bestored in directory storage service 220, etc.). For example, a givenclient 210 may include a suitable version of a web browser, or mayinclude a plug-in module or other type of code module that can executeas an extension to or within an execution environment provided by a webbrowser. Alternatively, a client 210 may encompass an application suchas a database application (or user interface thereof), a mediaapplication, an office application or any other application that maymake use of persistent storage resources to store and/or access one ormore hierarchical data structures to perform techniques likeorganization management, identity management, or rights/authorizationmanagement. In some embodiments, such an application may includesufficient protocol support (e.g., for a suitable version of HypertextTransfer Protocol (HTTP)) for generating and processing network-basedservices requests without necessarily implementing full browser supportfor all types of network-based data. That is, client 210 may be anapplication that can interact directly with network-based servicesplatform 200. In some embodiments, client 210 may generate network-basedservices requests according to a Representational State Transfer(REST)-style network-based services architecture, a document- ormessage-based network-based services architecture, or another suitablenetwork-based services architecture.

In some embodiments, a client 210 may provide access to network-basedservices to other applications in a manner that is transparent to thoseapplications. For example, client 210 may integrate with an operatingsystem or file system to provide storage in accordance with a suitablevariant of the storage models described herein. However, the operatingsystem or file system may present a different storage interface toapplications, such as a conventional file system hierarchy of files,directories and/or folders. In such an embodiment, applications may notneed to be modified to make use of the storage system service model.Instead, the details of interfacing to provider network 200 may becoordinated by client 210 and the operating system or file system onbehalf of applications executing within the operating systemenvironment.

Clients 210 may convey network-based services requests (e.g., accessrequests directed to hierarchical data structures in directory storageservice 220) to and receive responses from network-based servicesplatform 200 via network 260. In various embodiments, network 260 mayencompass any suitable combination of networking hardware and protocolsnecessary to establish network-based-based communications betweenclients 210 and platform 200. For example, network 260 may generallyencompass the various telecommunications networks and service providersthat collectively implement the Internet. Network 260 may also includeprivate networks such as local area networks (LANs) or wide areanetworks (WANs) as well as public or private wireless networks. Forexample, both a given client 210 and network-based services platform 200may be respectively provisioned within enterprises having their owninternal networks. In such an embodiment, network 260 may include thehardware (e.g., modems, routers, switches, load balancers, proxyservers, etc.) and software (e.g., protocol stacks, accounting software,firewall/security software, etc.) necessary to establish a networkinglink between given client 210 and the Internet as well as between theInternet and network-based services platform 200. It is noted that insome embodiments, clients 210 may communicate with network-basedservices platform 200 using a private network rather than the publicInternet.

Different types of hierarchical data structures may be stored, managed,and or represented in different ways. FIG. 4 is a block diagramillustrating one example of a data model for a hierarchal data storethat provides hierarchical data structures, according to someembodiments. An object may be the basic element of a hierarchical datastructure, such as directory structures 410 a or 410 n and may berepresented with circles or squares in the graph depicted of FIG. 4(e.g., objects 400, 401, 402, 403, 404, 405, 406, 407, and 421). Anobject may have a globally unique identifier (GUID), zero or moreattributes (key, value pairs), and zero or more links to other objects.In some embodiments, a directory may be one type of object which haszero or more child links to other objects, either directories orresources. Directory objects may have zero or one parent directoryobject, implying that directory objects and inks define a treestructure, in some embodiments. In FIG. 4A, object 401 is an example ofa directory object. Object 400 may be a root object that is the logicalroot multiple directory structures 410 and may not be visible to clientsof directory storage service 220. Resource objects (represented bysquares such as resource objects 404, 405, 406, and 407) may be leafobjects in a directory structure 410. A resource object may have aunique external Id (e.g., client specified) and client-definedattributes. Resource objects can have more than one parent object (whichwould allow for some hierarchical data structures to be configured as aDirected Acyclic Graph (DAG). Object 405 in FIG. 4 is an example of aresource object and it has two parents (objects 402 and 403).

In some embodiments, multiple types of resource objects may beimplemented. For example, in some embodiments, policy objects may be atype of resource object with two user-defined attributes: a policy typeand policy document (e.g., describing a policy applied to applicableobjects). For example, resource object 406 in FIG. 4 may be an exampleof a policy resource object. Another type of resource object may be anindex resource object. For example, an index resource object may be anindex on various attributes values of objects in the child objects andother descendant objects of the directory object to which the indexobject is attached. For example, if resource object 407 is an indexobject, then index object 407 may provide an index object for theattributes of child objects 402 and 403 as well as descendant objects404, 405, and 406.

In some embodiments, a link may be a directed edge between two objectsdefining a relationship between the two objects. There may be many typesof links, such as client visible link types and another link type forinternal operation implementation. In some embodiments, a child linktype may create a parent-child relationship between the objects itconnects. For example, child link ‘bb’ connects object 401 and object403. Child links may define the hierarchies of directory structures 410.Child links may be named in order to define the path of the object thatthe link points to. Another type of client visible link may be anattachment link. An attachment link may apply a resource object, such asa policy resource object or index resource object, to another resourceobject or directory object. Attachment links may not define thehierarchical structures of directory structures 410. For example,attachment link ‘xx’ applies the policy attribute stored in policyresource object 406 to directory object 402. Objects can have multipleattachments. In some embodiments, some attachment restrictions may beenforced, such as a restriction that not more than one policy resourceobject of any given policy type can be attached to a same object. Anon-client visible type of link or implied link type may also beimplemented in some embodiments, a reverse link. Reverse links may beused for optimizing traversal of directory structures 410 for commonoperations like resource object look-ups (e.g., policy lookups).Directory storage service 220 may maintain reverse links in the oppositedirection of child and attachment links.

In various embodiments, objects in directory structures 410 can beidentified and found by the pathnames that describe how to reach theobject starting from the logical root object 400, starting with the linklabeled “I” and following the child links separated by path separator“I” until reaching the desired object. For example, object 405 can beidentified using the path: “/directoryA/aa/dd”. As some objects may bechildren of multiple directory objects, multiple paths may identify anFor example, the following path can also be used to identify object 405:“/directoryA/bb/ee”. As directory structures 410 may be a collection ofobjects whose boundary is defined by the hierarchy of those objects inthe collection (e.g., the resulting hierarchical data structure, such asthe tree or DAG created by the links between objects). In this way,directory structures 410 may represent separate, independent, orpartially independent, organizations.

To store the illustrated directory structures in hierarchical datastructure store, the described objects, links attributes, and the likemay be modeled after a Resource Description Framework (RDF) data, insome embodiments. To maintain multiple versions of the hierarchical datastructures, versioning information may also be included to express howthe data has changed over time. RDF data may be structured as (Subject,Predicate, Object) tuples. When including additional versioninginformation this structure may become: (Subject, Predicate, Object,Version, PreviousVersion). To represent the hierarchical data structuresbased on RDF, there may be multiple types of RDF predicates. In someembodiments, one type of RDF predicates may represent links of thehierarchical data structure and another type of RDF predicates mayrepresent attributes of the hierarchical data structure. Different typesof predicts may represent the hierarchical data structure differently.Link predicates may be between two objects, whereas attribute predicatesmay be between an object and a value. Since a single object mightparticipate in several predicates of the same type, but with differentvalues, predicates may begin with a common prefix and end in someadditional type or naming information to aid in lookups. For example,the version entry in a tuple of a predicate may be the logical timestamp(e.g., transaction sequence number) at which the link or attribute wascreated, as all changes to a hierarchical data structure may utilize thetransaction resolution process provided by transaction log storage 250and may be assigned an ordered logical timestamp by transaction logstorage 250.

Storage nodes may maintain a current version of a hierarchical datastructure and past versions of a hierarchical data structure. In atleast some embodiments, different respective tables may be maintainedfor each hierarchical data structure, one table that stores the data forthe current version and another table that stores immutable records forthe previous versions. In various embodiments a current version table orprevious versions table may be accessed to perform various operationsfor a hierarchical data structure. For example an access request mayspecify a query: “Find all children for Object whose ID is GUID_401select GUID_401.child.* from CurrentVersion” or a query: “Find allpolicies for a resource object who's ID is GUID_405 along all paths tothe root. To service such queries, a depth first traversal may beexecuted along the parent links. At each object along the path to theroot, the following internal queries may be executed: internal query 1:“Find if the object has policies: select GUID_405.link.HasPolicy.* fromCurrentVersion;” internal query 2: “If the object has policies returnedin internal query 1, use the value from the link to get the policydocument value from the policy object: select GUID_406.link.PolicyDocfrom CurrentVersion;” internal query 3: “Find all parents for currentobject and perform internal queries 1-3 for each parent object untilreaching the root of the directory structure. Please note that previousexamples are not intended to be limiting as to the format, structure,syntax, or other ways in which queries may be expressed or processedwith respect to hierarchical data structures.

In at least some embodiments, a schema may be applied to a hierarchicaldata structure, such as a directory or sub-directory illustrated in FIG.4. FIG. 5 is a logical block diagram illustrating a data model for aschema applicable to a hierarchal data structure in a hierarchical datastore, according to some embodiments. Schema 500 may include one ormultiple schema classes, which may be organized according to inheritance(e.g., parent and child) links and named links. For example, schema 500in FIG. 5 illustrates six classes, 501, 503, 505, 507, 509, 511, 513 and515, which may be arranged according to inheritance links and namedlinks. A schema class may represent a set of attributes that can beattached to an object, in some embodiments. Each individual directoryobject may have any number of classes associated with it, and for eachof which class applied to the object may maintain its own independentset of attributes. Each schema class may consist of fundamentalmetadata, as illustrated at 520. The set of schema classes, theirconstraints, and the relationships between them may constitute a schema,such as schema 500.

A schema may be treated as a set of metadata which can be referred tofrom a variety of different locations (e.g., from different directorystructures). For example, a directory structure references a schema,that directory structure has “applied” the referenced schema. In atleast some embodiments, all data within a directory structure thatapplies a schema must conform to that schema. A schema can be applied tomultiple directories, serving as a blue-print for constructing andmaintaining the different directories. Once a schema is applied to adirectory structure, it may be extended or modified from the originallyreferenced form of the schema, as discussed below in some embodiments. Aschema may itself be modified so that each directory that applies theschema can apply the modified version of the schema (if desired), insome embodiments. Schema contents (e.g., classes, links, etc.) may beconsumed by programmatically by clients accessing the directory (e.g.,via API calls). By exploring the schema, clients can discover the fullset of classes, including attributes and constraints of the classes,which apply to data within the directory.

Inheritance links may define a parent or child schema class for a schemaclass. I some embodiments, a schema class may reference one or moreparent schema classes. Attributes may not be inherited from parentschema classes, in some embodiments. However, when an object is createdor associated with a schema class, the object may become an instance ofancestor schema classes as well, in some embodiments. For example, if a“Manager” class has an “Employee” class as a parent, and an “Employee”class has a “Person” class as a parent, assigning a new object to the“Manager” class may automatically assign the “Employee” and “Person” tothe object. This may allow for the effect of inheriting the attributesof the classes, but without those attributes crossing into the derivedclass's attribute set, in some embodiments. Attribute sets for eachclass may be permanently distinct. Inheritance links may be defined atschema class creation time, and may be immutable, in some embodiments.In this way, concrete assumptions about the presence and absence ofattributes for an object can be made, which facilitates modeling objecthierarchies, in various embodiments. Immutable class inheritance mayalso avoid scenarios where a parent class's attributes are either absent(in the case of a parent class added after the object was created) orstranded (in the case of a parent class removed after the object wascreated). In other embodiments, however, a schema classes may inheritattributes directly from parent schema classes.

Named links may allow one object of a class to hold a reference toanother object of another class within the hierarchical data structurewithout affecting policy inheritance, or implying organizationalownership between objects. Named links, in some embodiments, mayrepresent relationships, acting as an attribute which refers to anotherobject. An example is the relationship between an object of class“employee” and an object of class “laptop” assigned to the employee. Thelaptop may exist independently of the employee, and may have alternateparentage within the hierarchical data structure which definespermissions, cost centers, or other characteristics which may not beaffected by its relationship to the employee assigned to it. Therelationship may be navigable due to the named links from either side,but does not affect the application of policy to either item. In someembodiments, named links occupy the same namespace as normal attributesdefined in a class, that is, names of attributes and named links may notbe the same. A named link attribute may specify a target class. Thereferenced object may be an instance of the targeted class (or asub-class thereof). One form of a named link may be a one-to-manymapping (e.g., “user”). Setting the ‘user’ named link may override anyprevious setting for that link, essentially moving the attachment fromthe previously referenced object to the new object. In one embodiment,many-to-many mappings implemented as named link(s) may allow users tospecify numerous relations of a specific type, e.g., for deviceownership. Such mappings may act as a labelled set of children. Namedlinks may be independent of attachment between objects. Named links mayconnect any two objects regardless of type. Policy lookup may notconsider named links. Constraints on attachments made to objects may notapply to named links. For example, if an object of class ‘Employee’ canonly have other ‘Employee’ objects as children, defining a named linkthat links the ‘Employee’ to the ‘device’ may additionally allow deviceobjects to be attached to an ‘Employee’ that are not ‘Employee’ objects.

A class may be described by a set of class data, such that each objectthat is an instance of the class may include that data. For example,class data 520 may include a name, description (e.g., for developers,administrators, or consumers), constraints (e.g., an object that is aninstance of this class ‘must be child of an object that is an instanceof class X’, ‘child objects must be instances of class Y’),permission(s) (access controls for the object), attribute(s) (seeattributed data 530 below), named link(s) (as discussed above),inheritance (e.g., parent or child classes), and/or a tag, in someembodiments. One or multiple attributes may be included in a class, andmay include various data such as attribute data 530. In this way, aclass can describe what data is stored as part of an object and how thatdata may be interacted with by a client. For example, permissionsassigned to a schema class (or attribute or schema) may be consideredwhen requests are made against a schema (to read or apply it), oragainst an applied schema (to read or write directory data). Permissionsmay defined on the schema class, but influence all read operationsrelated to objects that are instances of that class. Consider a scenariowhere a client attempts to read a set of attribute fields on an object.The object may be an instance of a schema class and the schema class'spermissions may be included in the authorization of that request. Inthis way, a schema class's permissions may state a specific set ofclients (users or other entity associated with an identity) is allowedto read a specific list of attributes on any instances of that schemaclass. Similar permissions could be enforced with respect to otheroperations for objects that are instances of that schema class,including creation and deletion operations. Note that, in someembodiments, permissions may only apply to the set of attributes for aspecific schema class, and not for the directory object itself, which isan instance of the schema class. For instance, a client could be able toread and modify one set of attributes of an object that is an instanceof one schema class, and be unable to read another set of attributes onthe same object according to different permissions for the other set ofattributes of a different schema class (which the object is also aninstance of).

Attribute data 530 may include attribute name, description, type (e.g.,string, int, code, code pointer, etc.), execution data (see executiondata 540), indexed (e.g., a type of data that is indexed in thehierarchical data structure index), permissions (e.g., who canread/write the attribute data value), and default value (e.g., which maybe provided if not overwritten for a particular instance of the class).Different attribute data types may include, but are not limited to,String, Integer, Boolean, Decimal, and Date, to facilitate sorting orselecting objects based on inequality comparators (e.g., querying for abounded or unbounded interval in a range). In some embodiments,attributes may be defined as “shared”. Shared attributes may participatein a directory-global namespace which may be managed by schemamanagement 236 in FIG. 2. When two schema classes define sharedattributes with the same name, the values of those shared attributes maybe shared across the schema classes. For example, if a “User” schemaclass is defined with a shared “surname” attribute, and an “Employee”schema class attribute is also defined with a shared “surname”attribute, an update of one attribute will also update the other. Insome embodiments, only one value of a shared attributed may be stored.In order to update a shared attribute, a new value may have to bevalidated for all attribute definitions of the different class schemasthat share the attribute and apply to the object, in some embodiments.Attribute constraints may be automatically validated orvalidated/interacted with as part of execution data of client-specifiedcode, in some embodiments. Constraints may include min/max values,min/max lengths (e.g., for strings), acceptable character sets, orregular expression-based validation. In some embodiments, an attributemay be code that is triggered for execution when an object that is aninstance of the class that includes the attribute is accessed (e.g., forcreate, read, update, or delete).

Some classes, such as classes 509 and 511 may be application specificextensions that may only be read or otherwise accessed by theapplication that created them (or an application for which the creatingapplication granted access permissions). Permission(s) values in theclass data may indicate that the class information is applicationspecific by including an identifier or other value for determiningwhether or not an access request is from or otherwise associated withthe application that has privileges to access the attribute data of theclass. Alternatively, in some embodiments, application specificextensions may be made automatically available (e.g., for read access)to other applications. By allowing applications, to create and includeadditional classes in a schema, setting, configuring, or otherwisedetermining the various features of class data 520 and attribute data530 for the application, applications can enhance the functionality ofthe schema for application operations beyond storing additional data.For instance, by providing applications with flexibility to determineclass permissions, constraints, named links, and tags, applications canextend the operability of the schema, and thus the directory.

Tags, such as tag 540, may be included in some classes to performoperations with respect to objects using a collection of schema classes(including application specific classes) that are assigned or markedwith the same tag that group the classes together for performingdifferent operations, as discussed below with regard to FIGS. 10 and 11.For example, objects created using a tag may automatically become aninstance of the classes associated with the tag, even as that collectionof classes may change over time (as classed are added, like applicationspecific classes, or as classes are removed from the tag). In this way,an object can evolve over time as the schema evolves. For example, tagscan provide an application with defense against updates to the schemaover time, especially when accessing object which was created before theupdates.

Consider a scenario where an application relies upon a schema with a“PlayerData” class, intended to represent a player in the application'spublicly released game. The game is successful, and tens of thousands ofobjects are created using this class, and over time the applicationevolves to assume that “PlayerData” class is synonymous with aconceptual “player” in the game. In order to commoditize the game, moreinformation may be needed in the schema, such as a “PaymentInstrument”class for the players. Without tags, backfill operations would have tobe performed to include “PaymentInstrument” data on existing objects.Using tags, however, this problem can be avoided. Instead of creatinggeneric objects with a “PlayerData” class, the application can createobjects by tag (e.g., using an API such as client.createObjectByTag(“#player”)). In this way, an object is created which is tagged as a“player”, and consists of all classes for the player tag. When existingobjects, created prior to the “PaymentInstrument” application extension,are loaded for processing an access request, a request handler canidentify the “player” tag on the object and provide the existingPlayerData class and a PaymentInstrument class (consisting entirely ofdefault values for the attributes). Tags allow the decoupling of theconceptual intent (“create a player”) from the specific classes appliedto the object (“create an object with the PlayerData class”), therebyletting the application remain agnostic about future changes (which maybe added by other applications extending the schema).

New classes, including application specific classes that extend theschema, can be added to an existing tag, in various embodiments. In atleast some embodiments, adding a class to a tag may be limited toclasses that do not have any required attributes. In this way, abackfill operation may not have to be performed on existing objects. Theauthoring application of each schema class may determine which tags areassociated with it. Consider a scenario, where a new directory is set upusing a published schema, GamingPlatform version 1. An application(e.g., implementing a game that executes using the directory for thegaming platform) may extend the schema by creating a new class andtagging it with the standard “# player” tag, so that all players arecreated using the new class. Then, another version of GamingPlatformschema is published, version 2. Part of version 2 extends the “# player”definition further by adding a “PlayerExt” class. The new “PlayerExt”class will appear on all existing “# player” objects.

A schema may be stored in various ways. In some embodiments, a schemamay be stored as a document (e.g., a JSON document) in a standard,hierarchical structure or in another language independent data format.The document may contain all shape-related information for the schema,including attribute type, validation rules, indices, and other auxiliaryinformation. Indices, and information which loses context when movedbetween accounts (e.g., permissions, references to external resourceslike external code execution platforms, etc.), may be representedseparately in a manner which makes them easily distinguishable from thecore schema shape information (e.g., in a separate JSON document).Permissions for a schema may be stored outside of the document, in someembodiments. For example, schema classes may be represented as childrenof a “schema class” node attached to the root node, forming thestructural schema class hierarchy. Permissions may then be associatedwith each of the schema classes, in some embodiments, which can then beevaluated similar to perform a directory structure lookup operation asdiscussed above. Additional, clients can register for directory updateevent notifications without requiring mutation of the schema definition,in some embodiments.

FIG. 6 is a logical block diagram illustrating interactions to manage aschema applicable to a hierarchical data structure, according to someembodiments. A schema, as discussed above with regard to FIG. 5, may bemaintained in different states. In some embodiments, schema states mayinclude a schema in a working state, published state, or applied state.Different versions of the schemas in working, published, or appliedstates may also be maintained. In some embodiments, the interactionswith a schema may change dependent on the state of the schema beinginteracted with. For example, client 610 may send a request 640 tocreate a schema via interface 600 (e.g., via a graphical user interface(GUI), command line interface (CLI), and/or a programmatic interface(e.g., an API)). Schema management 236 may create 642 or allocate spacefor the schema in working schema store 620 (which may be stored on onemore storage nodes 240 or another data store). In the working state,client 610 may submit access requests to schema 650 which may includerequests to create a class, read a class, update a class, or delete aclass. Such access requests may be performed 642 by schema management236 with respect to the schema in working schema store 620. For example,attribute values for schemas (including code or pointers to code) may beadded or changed, as part of an access schema request 650.

Client 610 may submit a request to publish a schema 660 in workingstate. For example, the request may include an identifier for theschema, as well as a publication destination or access controls for thepublished schema. In some embodiments, schema management 236 may move orotherwise store 662 the schema in a published schema store 622 (whichmay be separate from working schema store 620 or located in a differentpart of the same data store as working schema store 620). Oncepublished, the access operations available to client 610 may change. Apublished schema, for example, may not be altered or modified. Asillustrated in FIG. 6, client 610 may access schema 670 and may includeoperations such as a request to read a class from the schema, whichschema management 236 may perform by accessing the schema 662.

Client 610 may submit a request to apply a schema 680 to a hierarchicaldata structure (e.g., such as a directory or sub-directory illustratedin FIG. 4). The request 680 may identify the hierarchical data structure(e.g., by identifier, name, or location). Schema management 236 maystore a copy of the published schema 642 to the applied schema store 630for the hierarchical data structure (e.g., which may be included in orco-located with a data store that stores the hierarchical datastructure, such as on the same storage node(s) that store a copy of thehierarchical data structure). Client 610 may also submit requests toaccess the schema 690, which may read classes in the applied schema orrequest modifications to the schema that extend the schema, as discussedabove with regard to FIGS. 1 and 5 and below with regard to FIGS. 8-11.For example, additional classes may be added, inheritance links added,or named links added, in some embodiments, that are specific to aparticular application. Modifications that remove or alter classes maybe added, in some embodiments, after the modifications are confirmed tonot conflict with the published schema that is the source of the appliedschema, as discussed below with regard to FIG. 9. For conflictingmodifications, the request to modify the schema (e.g., by anapplication) may blocked or denied, and may include an error responseindicating the conflict (e.g., providing the identity of the conflictingportions of the class), in some embodiments.

FIG. 7 is a logical block diagram illustrating interactions to access anobject in a hierarchical data structure that has application specificextensions to the schema, according to some embodiments. Storage node700, may be storage node like storage nodes 240 or 300 discussed abovewith regard to FIG. 3. Storage node 700 may implement request handler710, like request handler 310, to handle a request to access an object750 (e.g., dispatched to storage node by routing layer 232 in FIG. 2).Access object request 750 may identify the hierarchical data structure(e.g., by including a name or identifier), the object (e.g., byincluding a name, identifier, or location, such as a file path), and/orinformation indicating or describing the type of access request (e.g.,requests to create an object, read an object, update an object, ordelete an object), input parameters or other data to perform the accessrequest (e.g., schema class and attribute values for the object), and/oraccess credentials to access the hierarchical data structure or object.

Request handler 710 may receive the access request 750. Request handler710 may parse the access request to identify the schema. Request handler710 may then access the schema 762 in schema store 720 to retrieve theschema for application 764. For example, the access request may identifythe object (e.g., according to an identifier for the object or alocation, such as a directory path), the schema applied to thedirectory, the schema class, and the schema attribute to read or to reador update. Request handler 710 may access the schema, confirm that theobject is an instance of the schema class, and apply the schema class tothe object 764 (e.g., by identifying the name, type, format, or otherinformation for locating the attribute within hierarchical datastructure store 730). Request handler 710 may then access 764 the objectin hierarchical data structure store 730 to perform the request accessaccording to the applied schema. The object may then be returned inresponse to the access request (e.g., data values, etc.) or someindication of completion of the access request (e.g., acknowledgement ofan update or delete operation), which in turn may generate a response752 for access request 750 (e.g., indicating success, failure, requesteddata, etc.), in some embodiments. In at least some embodiments, requesthandler 710 may apply permissions specified in the identified schema(e.g., limiting access to a particular application) and/or may performvalidation of the operation (e.g., validating whether the access requestattempts to delete a “required” attribute value). Request handler 710may reject access requests 750 without permission to perform the accessrequest or that request an invalid operation, in some embodiments (e.g.,sending a response indicating an error code or exception).

The directory storage service, access requests, and other techniquesdiscussed in FIGS. 2 through 7 provide examples of a distributedhierarchical data store storing a hierarchical data structure for aclient and implementing application specific extensions to a schemaapplicable to a hierarchical data structure in different scenarios.However, various other types of hierarchical storage systems mayimplement extensions to a schema for a hierarchical data structure,which may utilize other numbers of types of components. FIG. 8 is ahigh-level flowchart illustrating methods and techniques to provideaccess to a hierarchical data structure with application specificextensions to a schema for the hierarchical data structure, according tosome embodiments. Various different distributed data stores includingthe embodiments described above may implement the techniques describedbelow.

As indicated at 810, a request may be received to access an object of ahierarchical data structure stored in a hierarchical data store, invarious embodiments, that is associated with one application out ofmultiple applications granted access to the hierarchical data store. Therequest may be received according to an interface, including a graphicaluser interface (GUI), command line interface (CLI), and/or aprogrammatic interface (e.g., an API), in one embodiment. The requestmay specify the hierarchical data structure (e.g., by name, identifier,or location). Similarly, the request may specify the object (e.g., byname, location, or identifier for an existing or to be created object).The request may also specify the schema class and/or attribute of theschema class to access, in some embodiments. As discussed above, withregard to FIG. 7, different types of access requests can be received.For example, in some embodiments, an access request to create theobject, read the object, update the object, or delete the object may bereceived.

As indicated at 820, a schema applicable to the hierarchical datastructure may be accessed, in some embodiments, to identify a class ofthe schema applied to the object. The class may be a class added to theschema after the application of the schema to the hierarchical datastructure by one of the applications granted access to the schema, insome embodiments. The access request may be received from theapplication that added the class or from a different application, insome embodiments. For example, as discussed above with regard to FIG. 6,different applications may access and extend the schema by submittingrequests to add a class to the schema. In some embodiments, the requestmay include an identifier for the schema and class, while in otherembodiments the mapping between applications and schemas and classes maybe maintained as part of a request handler or other component performingthe technique in FIG. 8. For example, a pointer, link, identifier, orother association mapping a schema to the hierarchical data structuremay be maintained as part of the hierarchical data structure so thatwhen the hierarchical data structure is accessed, the schema applied tothe hierarchical data structure is identified. The hierarchical datastructure may be stored in a different format than the schema, in someembodiments. For instance, the schema may be stored according tohuman-readable text format (e.g., Javascript Object Notation (JSON)),instead of in a hierarchical structure. The schema may be parsed,scanned, or otherwise consumed to determine what portions of the schemaare applicable to the object. For example, an inheritance determinationmay be performed to determine the class(es) of which the object is aninstance, according to inheritance information for each class includedin the schema. An object may be an instance of multiple differentclasses, but some of the classes may be specific to other schemas.

As indicated at 830, a determination may be made as to whether anapplication has permission to perform the access request, in variousembodiments. For example, in some embodiments, application added classesto a schema may be available when processing access requests from anyapplication granted access to the hierarchical data structure. Anapplication registry, for instance, may be implemented that allowsapplications to register and receive identity or authenticationcredentials that may be included in requests submitted by theapplication to the hierarchical data store so that access to ahierarchical data structure may be limited to registered applicationspresenting the proper credentials. In some embodiments, access rightsmay be fined grained, so that permission to perform an access requestmay be different depending on the identity of the application requestingthe access request. For example, permissions or other identity basedrestrictions for the access request may be imposed (e.g., as defined inthe class, as discussed above with regard to FIG. 5). For example,permissions can restrict access, providing limited (e.g., readprivileges) or no access to attribute values for an added class based onthe identity of the application. In some embodiments, defaultpermissions may be implemented so that, for instance, an applicationthat authored the class has read and write privileges for attributes inthe class, while other applications may be limited to read privileges.Thus, the permission determination indicated at 830 may evaluate theidentity of the application, the type of access request, or both todetermine whether the application has permission to perform theoperation. Other types of verification or validation operations for anaccess request (not illustrated) may be performed, such as determiningwhether the access request specifies a valid operation (e.g., anexisting class or attribute, a valid operation with respect to theattribute, such as deleting a required attribute value, etc.).

For those applications without permission, the request may be denied, asindicated at 850, in various embodiments. A denial response may be sent,in some embodiments, indicating a reason for denying the request (e.g.,lacking permission, invalid or mal-formed request, etc.). For thoseapplications with permission, the request to access the object may beperformed based on the identified class (e.g., updating an attributevalue in the class or reading an attribute value from the class).

FIG. 9 is a high-level flowchart illustrating methods and techniques toadd a class to a schema that extends the schema for an application thataccesses a hierarchical data structure, according to some embodiments.As indicated at 910, a request to add a class to a schema applied to ahierarchical data structure for processing access requests received froman application granted access to the hierarchical data structure may bereceived, in some embodiments. For example, the request may be receivedvia an interface, including a graphical user interface (GUI), commandline interface (CLI), and/or a programmatic interface (e.g., an API), inone embodiment. In some embodiments, the class may be included as partof the request, or may be uploaded or transmitted as part of asubsequent transmission. For example, a GUI may include a user interfaceelement allowing a user to manually enter the class data into a textediting space before submitting the class in the text editing space tobe included in the schema.

As indicated at 920, validity of the class for inclusion in the schemamay be evaluated, in some embodiments. For example, different extensionconstraints may be enforced with respect to application specific classesthat extend an applied schema. In one embodiment, constraint may beapplied on the format or type of attributes included in the class. Inone embodiment, a constraint may be applied on the naming convention ofthe class or attributes within the class. For example, a globalnamespace may be maintained with respect to class names so that no classin a schema may be named the same as another class. Similarly, a schemamay reserve some words so that they cannot be used as a class name orattribute value in some embodiments. In one embodiment, a constraint maybe applied on the inheritance of the class (e.g., one applicationspecific extensions cannot inherit from another application specificextension). Various combinations of constraints may be applied orenforced, in different embodiments. Thus, the previous examples are notintended to be limiting. Validation may also be performed so that theaddition of the schema class does not interfere with the operation ofany other applications that utilize the schema as part of accessing thehierarchical data structure. For example, a constraint may beimplemented so that a class is backward compatible with all otherpre-existing classes (including pre-existing extensions already includedby the same and/or other applications). The backward compatibleconstraint may include conditions such as only allowing additionalclasses that include attributes that are not mandatory or required(e.g., can have null values or default values generated).

As indicated by the negative exit from 920, the request to include aclass that it is determined to be invalid may be denied, as indicated at940. For example, a response indicating request failure may be sent. Inat least some embodiments, the response may include or indicatevalidation errors (to help identify corrections that could be performedto make the class valid). As indicated by the positive exit from 920, aclass that is determined to be valid may be stored for inclusion in theschema. In some embodiments, the schema document or data structure maybe directly updated. For example, a JSON storing the schema or otherdocument in the applied schema store may be updated directly. In someembodiments an additional or addendum document or data object may becreated. In some embodiments, the schema may be stored as hierarchicaldata structure, and thus inclusion of the class may include creating anew object the hierarchical data structure of the schema.

Tags, as discussed above with regard to FIG. 5, may be applied to groupor collect classes together for performing different operations in ahierarchical data structure. For example, operations that are invokedusing a tag can dynamically determine what classes are applicable andthen perform the operation based on the identified classes. In this way,class creators can easily add new classes that can be accessible toexisting objects by associating the new classes with a tag for a classthat the objects are already instances of, instead of manually applyingthe new classes to each existing object. FIG. 10 is a high-levelflowchart illustrating methods and techniques to update an object basedon a tag, according to some embodiments.

As indicated at 1010, a request may be received to add a class to aschema for a hierarchical data structure that indicates a tag for theclass. Tags may be publicly advertised or accessible when accessing adirectory, by accessing the schema data for a class (e.g., asillustrated in FIG. 5 above). Tags may also be created for a specificapplication (e.g., by sending a request to create a tag for a schema orby including the tag in a class creation or object creation request). Asindicated at 1020, the schema may be updated to include the class. Forexample, a JSON storing the schema or other document in the appliedschema store may be updated directly, or an additional or addendumdocument or data object may be created. In some embodiments, the schemamay be stored as hierarchical data structure, and thus inclusion of theclass may include creating a new object the hierarchical data structureof the schema. As part of updating the schema, the data of the class mayinclude the tag indicated in the request at 1010.

In addition to associating with classes with tags, objects may also beassociated with a tag, in some embodiments. For example, as discussedbelow with regard to FIG. 11, an object can be created according to atag, including the tag as part of the object. In some embodiments, arequest to “tag” an object with a tag may be received and performed sothat the object includes the tag (e.g., in object data or metadata inthe hierarchical data structure). Operations, such as access requests,can then be performed according to the tag. As indicated at 1030, in atleast some embodiments, an access request may be received for an object.When processing the access request, a determination may be made, asindicated at 1040, as to whether the object includes a tag (e.g., thetag indicated at 1010), in various embodiments. For example, consider ascenario where the added class is a “Player Payer Instrument Class” thatindicates the tag “player” and the object targeted by the access requestincludes the tag “player.” A lookup or evaluation of the existing objectmay identify the tag as included in or associated with the object.

If the object includes the tag, then as indicated at 1050 the class maybe included as applied to the object for processing the access request(e.g., along with any other classes that were manually applied using arequest to apply the class to the object). Including the class may makethe class, and thus attributes of the class, available for processingthe access request. Thus, when processing the request based on otherclass(es) applied to the object, as indicated at 1060, the attribute ofthe classes may be available to serve the access request. For example,if the request attempts to update a value for an attribute in “PlayerPayer Instrument Class” then the access request would succeed eventhough the “Player Payer Instrument Class” was never explicitly appliedvia a manual request to the object. If the object, were not to includethe tag, then only those classes applied to the object (e.g., manuallyor via another tag) would be applicable for processing the accessrequest, as indicated by the negative exit from 1040. Thus, in theprevious example, the access request for the attribute value in “PlayerPayer Instrument Class” would fail as that class has not been applied tothe object.

FIG. 11 is a high-level flowchart illustrating methods and techniques tocreate an object in a hierarchical data structure according to a tag,according to some embodiments. As indicated at 1110, a request may bereceived to create an object in a hierarchical data structure accordingto a tag. For example, an API or other interface may allow a creationrequest to include a tag identifier (as a field or flag within therequest) or may include a specific tag interface to create an objectaccording to a tag. As discussed above, tags may be advertised orotherwise provided for a schema as part of a published version of aschema. Like extensions to the schema, new tags can be created that arespecific to the application that created the tag, allowing for tag-basedcreation (or updates as discussed above with regard to FIG. 10) to beperformed using tags that are only visible to and under the control ofone application. In other embodiments, tags may be available to anyapplication granted access to the hierarchical data structure.

As indicated at 1120, class(es) of the schema that include the tag maybe identified. For example, an index, list, or other data structure maybe maintained for each tag that is utilized in the schema in order toperform a quick look up to find classes that include a desired tag. Insome embodiments, the schema may be parsed, traversed, scanned, orotherwise evaluated to identify the class(es) associated with a tag. Asindicated at 1130, the object may then be created in the hierarchicaldata structure according to the identified classes, in some embodiments.For example, default values for attributes defined in the class(es) maybe included and/or space allocated in the object to store subsequentlyreceived values for the attributes of the class(es). The object may alsobe marked with or otherwise include the tag in order to perform othertag-based operations, such as applying subsequently added schema classesas discussed above with regard to FIG. 10.

The methods described herein may in various embodiments be implementedby any combination of hardware and software. For example, in oneembodiment, the methods may be implemented by a computer system (e.g., acomputer system as in FIG. 12) that includes one or more processorsexecuting program instructions stored on a computer-readable storagemedium coupled to the processors. The program instructions may implementthe functionality described herein (e.g., the functionality of variousservers and other components that implement the directory storageservice and/or storage services/systems described herein). The variousmethods as illustrated in the figures and described herein representexample embodiments of methods. The order of any method may be changed,and various elements may be added, reordered, combined, omitted,modified, etc.

FIG. 12 is a block diagram illustrating a computer system that canimplement the distributed data store application specific schemaextensions for a hierarchical data structure, according to variousembodiments, as well as various other systems, components, services ordevices described above. For example, computer system 2000 may implementhierarchy storage nodes that maintain versions of hierarchical datastructures or components of a transaction log store that maintaintransaction logs for hierarchical data structures, in differentembodiments. Computer system 2000 may be any of various types ofdevices, including, but not limited to, a personal computer system,desktop computer, laptop or notebook computer, mainframe computersystem, handheld computer, workstation, network computer, a consumerdevice, application server, storage device, telephone, mobile telephone,or in general any type of computing device.

Computer system 2000 includes one or more processors 1010 (any of whichmay include multiple cores, which may be single or multi-threaded)coupled to a system memory 1020 via an input/output (I/O) interface1030. Computer system 2000 further includes a network interface 1040coupled to I/O interface 1030. In various embodiments, computer system2000 may be a uniprocessor system including one processor 1010, or amultiprocessor system including several processors 1010 (e.g., two,four, eight, or another suitable number). Processors 1010 may be anysuitable processors capable of executing instructions. For example, invarious embodiments, processors 1010 may be general-purpose or embeddedprocessors implementing any of a variety of instruction setarchitectures (ISAs), such as the x86, PowerPC, SPARC, or MIPS ISAs, orany other suitable ISA. In multiprocessor systems, each of processors1010 may commonly, but not necessarily, implement the same ISA. Thecomputer system 2000 also includes one or more network communicationdevices (e.g., network interface 1040) for communicating with othersystems and/or components over a communications network (e.g. Internet,LAN, etc.). For example, a client application executing on system 2000may use network interface 1040 to communicate with a server applicationexecuting on a single server or on a cluster of servers that implementone or more of the components of the directory storage systems describedherein. In another example, an instance of a server applicationexecuting on computer system 2000 may use network interface 1040 tocommunicate with other instances of the server application (or anotherserver application) that may be implemented on other computer systems(e.g., computer systems 1090).

In the illustrated embodiment, computer system 2000 also includes one ormore persistent storage devices 1060 and/or one or more I/O devices1080. In various embodiments, persistent storage devices 1060 maycorrespond to disk drives, tape drives, solid state memory, other massstorage devices, or any other persistent storage device. Computer system2000 (or a distributed application or operating system operatingthereon) may store instructions and/or data in persistent storagedevices 1060, as desired, and may retrieve the stored instruction and/ordata as needed. For example, in some embodiments, computer system 2000may host a storage system server node, and persistent storage 1060 mayinclude the SSDs attached to that server node.

Computer system 2000 includes one or more system memories 2020 that canstore instructions and data accessible by processor(s) 2010. In variousembodiments, system memories 2020 may be implemented using any suitablememory technology, (e.g., one or more of cache, static random accessmemory (SRAM), DRAM, RDRAM, EDO RAM, DDR 10 RAM, synchronous dynamic RAM(SDRAM), Rambus RAM, EEPROM, non-volatile/Flash-type memory, or anyother type of memory). System memory 2020 may contain programinstructions 2025 that are executable by processor(s) 2010 to implementthe methods and techniques described herein. In various embodiments,program instructions 1025 may be encoded in platform native binary, anyinterpreted language such as Java™ byte-code, or in any other languagesuch as C/C++, Java™, etc., or in any combination thereof. For example,in the illustrated embodiment, program instructions 1025 include programinstructions executable to implement the functionality of a hierarchystorage nodes that maintain versions of hierarchical data structures orcomponents of a transaction log store that maintain transaction logs forhierarchical data structures, in different embodiments. In someembodiments, program instructions 1025 may implement multiple separateclients, server nodes, and/or other components.

In some embodiments, program instructions 1025 may include instructionsexecutable to implement an operating system (not shown), which may beany of various operating systems, such as UNIX, LINUX, Solaris™, MacOS™,Windows™, etc. Any or all of program instructions 1025 may be providedas a computer program product, or software, that may include anon-transitory computer-readable storage medium having stored thereoninstructions, which may be used to program a computer system (or otherelectronic devices) to perform a process according to variousembodiments. A non-transitory computer-readable storage medium mayinclude any mechanism for storing information in a form (e.g., software,processing application) readable by a machine (e.g., a computer).Generally speaking, a non-transitory computer-accessible medium mayinclude computer-readable storage media or memory media such as magneticor optical media, e.g., disk or DVD/DIRECTORY STORAGE SERVICE 220-ROMcoupled to computer system 2000 via I/O interface 1030. A non-transitorycomputer-readable storage medium may also include any volatile ornon-volatile media such as RAM (e.g. SDRAM, DDR SDRAM, RDRAM, SRAM,etc.), ROM, etc., that may be included in some embodiments of computersystem 2000 as system memory 1020 or another type of memory. In otherembodiments, program instructions may be communicated using optical,acoustical or other form of propagated signal (e.g., carrier waves,infrared signals, digital signals, etc.) conveyed via a communicationmedium such as a network and/or a wireless link, such as may beimplemented via network interface 1040.

In some embodiments, system memory 1020 may include data store 1045,which may be implemented as described herein. For example, theinformation described herein as being stored by the hierarchy storagenodes or transaction log store described herein may be stored in datastore 1045 or in another portion of system memory 1020 on one or morenodes, in persistent storage 1060, and/or on one or more remote storagedevices 1070, at different times and in various embodiments. In general,system memory 1020 (e.g., data store 1045 within system memory 1020),persistent storage 1060, and/or remote storage 1070 may store datablocks, replicas of data blocks, metadata associated with data blocksand/or their state, database configuration information, and/or any otherinformation usable in implementing the methods and techniques describedherein.

In one embodiment, I/O interface 1030 may coordinate I/O traffic betweenprocessor 1010, system memory 1020 and any peripheral devices in thesystem, including through network interface 1040 or other peripheralinterfaces. In some embodiments, I/O interface 1030 may perform anynecessary protocol, timing or other data transformations to convert datasignals from one component (e.g., system memory 1020) into a formatsuitable for use by another component (e.g., processor 1010). In someembodiments, I/O interface 1030 may include support for devices attachedthrough various types of peripheral buses, such as a variant of thePeripheral Component Interconnect (PCI) bus standard or the UniversalSerial Bus (USB) standard, for example. In some embodiments, thefunction of I/O interface 1030 may be split into two or more separatecomponents, such as a north bridge and a south bridge, for example.Also, in some embodiments, some or all of the functionality of I/Ointerface 1030, such as an interface to system memory 1020, may beincorporated directly into processor 1010.

Network interface 1040 may allow data to be exchanged between computersystem 2000 and other devices attached to a network, such as othercomputer systems 1090 (which may implement embodiments describedherein), for example. In addition, network interface 1040 may allowcommunication between computer system 2000 and various I/O devices 1050and/or remote storage 1070. Input/output devices 1050 may, in someembodiments, include one or more display terminals, keyboards, keypads,touchpads, scanning devices, voice or optical recognition devices, orany other devices suitable for entering or retrieving data by one ormore computer systems 2000. Multiple input/output devices 1050 may bepresent in computer system 2000 or may be distributed on various nodesof a distributed system that includes computer system 2000. In someembodiments, similar input/output devices may be separate from computersystem 2000 and may interact with one or more nodes of a distributedsystem that includes computer system 2000 through a wired or wirelessconnection, such as over network interface 1040. Network interface 1040may commonly support one or more wireless networking protocols (e.g.,Wi-Fi/IEEE 802.11, or another wireless networking standard). However, invarious embodiments, network interface 1040 may support communicationvia any suitable wired or wireless general data networks, such as othertypes of Ethernet networks, for example. Additionally, network interface1040 may support communication via telecommunications/telephony networkssuch as analog voice networks or digital fiber communications networks,via storage area networks such as Fibre Channel SANs, or via any othersuitable type of network and/or protocol. In various embodiments,computer system 2000 may include more, fewer, or different componentsthan those illustrated in FIG. 12 (e.g., displays, video cards, audiocards, peripheral devices, other network interfaces such as an ATMinterface, an Ethernet interface, a Frame Relay interface, etc.)

It is noted that any of the distributed system embodiments describedherein, or any of their components, may be implemented as one or morenetwork-based services. For example, a database engine head node withinthe database tier of a database system may present database servicesand/or other types of data storage services that employ the distributedstorage systems described herein to clients as network-based services.In some embodiments, a network-based service may be implemented by asoftware and/or hardware system designed to support interoperablemachine-to-machine interaction over a network. A network-based servicemay have an interface described in a machine-processable format, such asthe Web Services Description Language (WSDL). Other systems may interactwith the network-based service in a manner prescribed by the descriptionof the network-based service's interface. For example, the network-basedservice may define various operations that other systems may invoke, andmay define a particular application programming interface (API) to whichother systems may be expected to conform when requesting the variousoperations.

In various embodiments, a network-based service may be requested orinvoked through the use of a message that includes parameters and/ordata associated with the network-based services request. Such a messagemay be formatted according to a particular markup language such asExtensible Markup Language (XML), and/or may be encapsulated using aprotocol such as Simple Object Access Protocol (SOAP). To perform anetwork-based services request, a network-based services client mayassemble a message including the request and convey the message to anaddressable endpoint (e.g., a Uniform Resource Locator (URL))corresponding to the network-based service, using an Internet-basedapplication layer transfer protocol such as Hypertext Transfer Protocol(HTTP).

In some embodiments, network-based services may be implemented usingRepresentational State Transfer (“RESTful”) techniques rather thanmessage-based techniques. For example, a network-based serviceimplemented according to a RESTful technique may be invoked throughparameters included within an HTTP method such as PUT, GET, or DELETE,rather than encapsulated within a SOAP message.

The various methods as illustrated in the figures and described hereinrepresent example embodiments of methods. The methods may be implementedmanually, in software, in hardware, or in a combination thereof. Theorder of any method may be changed, and various elements may be added,reordered, combined, omitted, modified, etc.

Although the embodiments above have been described in considerabledetail, numerous variations and modifications may be made as wouldbecome apparent to those skilled in the art once the above disclosure isfully appreciated. It is intended that the following claims beinterpreted to embrace all such modifications and changes and,accordingly, the above description to be regarded in an illustrativerather than a restrictive sense.

1-20. (canceled)
 21. A system, comprising: at least one processor; and amemory, storing program instructions that when executed by the at leastone processor, cause the at least one processor to implement a directoryservice, the directory service configured to: receive a request tocreate an extension to a schema on an object of a directory hosted bythe directory service, wherein the request is to create the extensionfor a first application; create the requested extension to the schema onthe object of the directory for the first application; receive a requestto access the object for a second application; determine that the secondapplication has permission to access the extension to the schema on theobject; and perform the request to access the object for the secondapplication.
 22. The system of claim 21, wherein the directory serviceis further configured to: receive a second request to access the objectfor a third application; determine that the third application does nothave permission to access the extension to the schema on the object; anddeny the second request to access the object for the third application.23. The system of claim 21, wherein the permission to access theextension is limited to a read.
 24. The system of claim 21, wherein theaccess request is a read request.
 25. The system of claim 21, whereinthe access request is a write request.
 26. The system of claim 21,wherein the directory service is further configured to delete theextension to the schema on the object responsive to a request to deletethe extension.
 27. The system of claim 21, wherein the directory serviceis offered as part of a provider network, wherein the first applicationand the second application are implemented on one or more other servicesoffered by the provider network.
 28. A method, comprising: receiving, ata directory service, a request to create an extension to a schema on anobject of a directory hosted by the directory service, wherein therequest is to create the extension for a first application; creating, bythe directory service, the requested extension to the schema on theobject of the directory for the first application; receiving, at thedirectory service, a request to access the object for a secondapplication; determining, by the directory service, that the secondapplication has permission to access the extension to the schema on theobject; and performing, by the directory service, the request to accessthe object for the second application.
 29. The method of claim 28,further comprising: receiving, at the directory service, a secondrequest to access the object for a third application; determining, bythe directory service, that the third application does not havepermission to access the extension to the schema on the object; anddenying, by the directory service, the second request to access theobject for the third application.
 30. The method of claim 28, whereinthe permission to access the extension is limited to a read.
 31. Themethod of claim 28, wherein the access request is a read request. 32.The method of claim 28, wherein the access request is a write request.33. The method of claim 28, further comprising deleting the extension tothe schema on the object responsive to a request to delete theextension.
 34. The method of claim 28, wherein the directory service isoffered as part of a provider network, wherein the first application andthe second application are implemented on one or more other servicesoffered by the provider network.
 35. One or more non-transitory,computer-readable storage media, storing program instructions that whenexecuted on or across one or more computing devices, cause the one ormore computing devices to implement: receiving, at a directory service,a request to create an extension to a schema on an object of a directoryhosted by the directory service, wherein the request is to create theextension for a first application; creating, by the directory service,the requested extension to the schema on the object of the directory forthe first application; receiving, at the directory service, a request toaccess the object for a second application; determining, by thedirectory service, that the second application has permission to accessthe extension to the schema on the object; and performing, by thedirectory service, the request to access the object for the secondapplication.
 36. The one or more non-transitory, computer-readablestorage media of claim 35, storing further instructions that whenexecuted on or across the one or more computing devices cause the one ormore computing devices to further implement: receiving, at the directoryservice, a second request to access the object for a third application;determining, by the directory service, that the third application doesnot have permission to access the extension to the schema on the object;and denying, by the directory service, the second request to access theobject for the third application.
 37. The one or more non-transitory,computer-readable storage media of claim 35, wherein the permission toaccess the extension is limited to a read.
 38. The one or morenon-transitory, computer-readable storage media of claim 35, wherein theaccess request is a read request.
 39. The one or more non-transitory,computer-readable storage media of claim 35, wherein the access requestis a write request.
 40. The one or more non-transitory,computer-readable storage media of claim 35, storing furtherinstructions that when executed on or across the one or more computingdevices cause the one or more computing devices to further implementdeleting the extension to the schema on the object responsive to arequest to delete the extension.